System and method of identifying web page semantic structures

ABSTRACT

The disclosure presents a method, system and computer-readable medium related to automatically analyzing structure for a web page. The method embodiment comprises building a training corpus comprising a broad stylistic coverage of web pages, segmenting a web page into information blocks, identifying semantic categories of the information blocks using the training corpus and applying the identical semantic categories in a web-based tool.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a system and method of identifying webpage semantic structures.

2. Introduction

In spite of recent progress on the semantic web and interchange formatslike XML, most available web pages today are still written in HTML anddesigned mainly for humans and not machines to read. Informationconveyed on HTML pages is carried not only by their stream of texts, butalso by the layout of the web pages. For instance, the web page in FIG.2 consists of a form and a horizontal menu on the top, a heading-contentvisual block and a vertical menu on the left, and severalheading-content and normal-content descriptions in the center. Humanscan easily recognize this structure by following visual clues andlanguage clues. A variety of web-based applications have begun toexploit web page semantic structures. For example, web page layoutextraction is a fundamental component of AT&Ts WebTalk, which is aframework for automatically constructing dialog systems using companywebsites. Others have used web page semantic structures for adaptivelydisplaying web pages on small devices or to build a domain specificproduct extraction system such as DataRover, which is based on a webpage segmentation algorithm.

However, automatically recognizing web page semantic structures is by nomeans an easy task. An HTML developer can choose using templates, whitespaces, images, tables, dozens of HTML tags, hundreds of HTMLattributes, or a combination of them to artistically lay out a page.HTML source codes for rendering the same web page could be dramaticallydifferent from one developer to another.

The Document Object Model (DOM) is widely used as the representationmodel of HTML documents. FIG. 3 shows a DOM tree fragment for the webpage in FIG. 2. Several DOM-based heuristic algorithms have beendeveloped for discovering the semantic structures of web pages. Thesealgorithms are initiated by two key observations, which are:

First, contiguous leaf nodes on the DOM tree are semantically related ifthey have similar root-to-leaf tag paths. Based on this observation,researchers have developed a web page segmentation algorithm that takesthe DOM tree as input and collects the root-to-leaf tag-path for eachleaf node on the tree. A segment boundary is found, when the tag-pathsimilarity between two contiguous leaf nodes is below a predefinedthreshold 6. Based on the same observation, a more complex algorithm hasbeen proposed to group the leaf nodes in the DOM into a semanticpartition tree. See, Saikat Mukerjee, GuiZhen Yang, WenFang Tan, I. V.Ramakrishman, “Automatic Discovery of Semantic Structures in HTMLDocuments”, ICDAR 2003, incorporated herein by reference.

Second, semantic blocks on a web page are often separated by visualseparators such as lines, blank areas, images, font sizes, colors, etc.A Vision-based Page Segmentation (VIPS) algorithm has been proposed todetect the semantic content structure in a web page. VIPS makes use ofthe DOM structure as well as visual cues of DOM tree nodes includingposition, color, font size, font weight, etc. A list of heuristic rulesare implemented to determine the visual blocks. An example of such arule is dividing a DOM node if its background color is different fromone of its children's.

These observations and the above discussed algorithms addressed part ofthe web page authors' intents for layout presentation. However, thereare much more complex and richer cues remaining unemployed butintensively encoded by web page authors and used by web page readerssuch as language features, geometric cues, miscellaneous HTMLattributes, etc. As a result, the effectiveness of these algorithms isonly limited to some circumstances.

An HTML document is encoded through HTML tags (such as “<font>”),attributes (such as “color”), attribute values (such as“color=#003355”), as well as text (such as “ZOOM VARplus Program” inFIG. 2). HTML 4.01 Specification, incorporated herein by reference aswell as updates to the HTML protocol, specifies 91 HTML tags and 119attributes. They are used to govern the structure, the presentation ofthe rendered web page as well as the interactivity with the web page.Given this complexity, it is difficult if not impossible to develop aheuristic algorithm that is able to appropriately take into account thislarge number of contributing factors for the web page semanticstructure.

INVENTION SUMMARY

This application discusses a machine learning approach and the featuresthat are used for web page segmentation and web page information blockclassification. Experiments and experimental results are presented.

The present invention addresses the issues in using machine learningtechniques to treat web page semantic structures as a task involving webpage segmentation and information block classification. An informationblock is defined as a coherent topic area according to its content or acoherent functional area according to its associated behavior. An aspectof the invention is to define a group of semantic categories (e.g. 12categories) for classifying web page information blocks into includingPage-Title, Form, Table-Data, FAQ-Answer, Menu, Bulletined-List,Heading, Heading-List, Normal-Content, Heading-Content, Picture-Label,and Other. FIG. 2 illustrates this task with an example where a web pageis segmented into information blocks. In this example, there are eightdifferent types of segmented blocks. The segmentation may be a singleblock or any plurality of blocks and each of these blocks is assigned acategory. Such an analysis can benefit a wide spectrum of web-basedapplications including search engines, question answering and datamining systems. For example, a search engine can potentially retrievebetter search results by exploiting these semantic categories andapplying different weights to them.

The disclosure presents a method, system and computer-readable mediumrelated to automatically analyzing structure for a web page. The methodembodiment comprises building a training corpus comprising a broadstylistic coverage of web pages, segmenting a web page into informationblocks, identifying semantic categories of the information blocks usingthe training corpus and applying the identical semantic categories in aweb-based tool.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific embodiments thereof which areillustrated in the appended drawings. Understanding that these drawingsdepict only typical embodiments of the invention and are not thereforeto be considered to be limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1A illustrates the basic communication between a client and aserver;

FIG. 1B illustrates the basic components of a computing device for usewith the invention;

FIG. 2 illustrates an example web page;

FIG. 3 illustrates an example DOM tree arrangement; and

FIG. 4 illustrates a method embodiment of the invention.

DETAILED DESCRIPTION

Various embodiments of the invention are discussed in detail below.While specific implementations are discussed, it should be understoodthat this is done for illustration purposes only. A person skilled inthe relevant art will recognize that other components and configurationsmay be used without parting from the spirit and scope of the invention.

FIG. 1A illustrates a simple arrangement of a client computing device102 communicating through a network 104 with a computing server 106.Each of these components is known to those of skill in the art.Variations on this basic arrangement are also known—such as the abilityof the network 104 to be a cellular network, telephone network, cablenetwork, any type of wireless network such as a Bluetooth based network,or any other type of wireless communication network. The computingdevice 102 may be a desktop computer, laptop computer, portablecomputing device, smartphone, or any other type of computing device. Theserver 106 is typically a server upon which a website operates and isused to communicate web pages to the client 102. The basic functionalityof the invention is independent of the type of computing device ornetwork through which computing devices communicate.

FIG. 1B illustrates an exemplary processing system 100 upon whichimplementations consistent with the principles of the invention mayexecute. System 100 may include a bus 110, a processor 120, a memory130, a read only memory (ROM) 140, a storage device 150, an input device160, an output device 170, and a communication interface 180. Bus 110may permit communication among the components of system 100.

Processor 120 may include at least one conventional processor ormicroprocessor that interprets and executes instructions. Memory 130 maybe a random access memory (RAM) or another type of dynamic storagedevice that stores information and instructions for execution byprocessor 120. Memory 130 may also store temporary variables or otherintermediate information used during execution of instructions byprocessor 120. ROM 140 may include a conventional ROM device or anothertype of static storage device that stores static information andinstructions for processor 120. Storage device 150 may include any typeof media, such as, for example, magnetic or optical recording media andits corresponding drive. Storage device 150 may include a database thatincludes a number of records including strings that may be compared withone or more search strings in implementations consistent with theprinciples of the invention.

Input device 160 may include one or more conventional mechanisms thatpermit a user to input information to system 100, such as a keyboard, amouse, a pen, a voice recognition device, etc. Output device 170 mayinclude one or more conventional mechanisms that output information tothe user, including a display, a printer, one or more speakers, or amedium, such as a memory, or a magnetic or optical disk and acorresponding disk drive. Communication interface 180 may include anytransceiver-like mechanism that enables system 100 to communicate via anetwork. For example, communication interface 180 may include a modem,or an Ethernet interface for communicating via a local area network(LAN). Alternatively, communication interface 180 may include othermechanisms for communicating with other devices and/or systems viawired, wireless or optical connections.

System 100 may perform such functions in response to processor 120executing sequences of instructions contained in a computer-readablemedium, such as, for example, memory 130, a magnetic disk, or an opticaldisk. Such instructions may be read into memory 130 from anothercomputer-readable medium, such as storage device 150, or from a separatedevice via communication interface 180.

Web page segmentation is a task to group text nodes on a web page into asequence of information blocks. This task may be characterized as abinary classification problem. An exemplary web page 200 from AT&T isorganized for a user to see as is shown in FIG. 2. Features includelinks 210 with options to enter search terms 212, or an AT&T keyword 214with a search field 216 for the user to insert text. Other optionsinclude choosing links to other pages such as a local voice service 218,products and services 220, local solutions 222, and so forth in box202A. A larger title is highlighted as 208 and other boxes 204, 206A,206B, 206C and 202B show different groupings of text in the web page200. For each pair of contiguous text nodes (such as “Enter” and “SearchTerms” in FIG. 2), a set of features is built to represent the distanceand difference between them, and then classify this feature set into theinformation block boundary class or the non-boundary class. These twonodes are separated into two information blocks, if a boundary isidentified between them.

Web page information block classification is, for example, a 12-classclassification task. The class may be zero, one or a plurality-classclassification. Adaboost and SVMs are two popular machine learningalgorithms for the problem of classification. Both of them are capableof inferring regularities using many heterogeneous features andgeneralizing without requiring the careful feature selection. Thesecapabilities are just what we need in this learning task, since web pagestructures are characterized by a large number of features and it is notclear which of them are efficiently contributing.

Training examples can be represented as: (x₁, y₁), . . . (x_(i), y_(i)),. . . (x_(N), y_(N)), where each x_(i) is a feature set belonging to aspace X, and each y_(i) is in some label set Y. Adaboost calls a givenweak learning algorithm repeatedly in a series of rounds t=1, . . . , T.Each time the weak learning algorithm is applied, it generates a roughrule of thumb. And after many rounds Adaboost combines these weak rulesinto a single prediction rule that is typically more accurate onpredicting y_(i) than any one of the weak rules.

Those of skill in the art will be familiar to Adaboost and otherAdaboost-based algorithms. The preferred system is Boostexter, anAdaboost-based system for text categorization, as the boosting-basedlearning and testing tool. Boostexter has been widely used and proved tobe a state-of-the-art technique for text classification. It works withdata which may be of various forms. In general, each instance is brokeninto multiple fields. These fields may be of four types: acontinuous-valued attribute (such as “age”), a discrete-valued attribute(such as “eye color”), a text string (such as “body of an emailmessage”), and a scored text string (in which scores are associated witheach word of the text). Other types may also be identified and utilized.

SVM is a kernel-based learning algorithm for optimizing separating hyperplanes between y_(i). A kernel is a function that maps the input data Xinto a high dimensional feature space. The preferable algorithm for SVMtraining is the Llama tool as the SVM training and testing tool. Llamais an implementation of multi-class SVMs with heterogeneous kernels. Itallows the user to specify a list of hypotheses kernels to explore.Validation sets are set apart automatically and used to determine theoptimal kernel to learn each class against all others. Llama works withdata of continuous and discrete values.

The advantage for applying Machine Learning to this task is that itenables one to make use of a large number of heterogeneous featureswithout clearly knowing which of them are effectively contributing andhow they collaborate with each other.

Given that both web page segmentation and information blockclassification focus on processing texts, the inventors attempted tocollect all related features for each text node t_(i), where i is thenumber of the text node on the DOM tree ordered from left to right.Related features may include a wide variety of parameters and include atleast a representation mode structure feature, a geometric feature, anHTML attribute, neighboring nodes and linguistic features. Examples ofthese features are given and explained in the following:

DOM tree structure features including the root-to-leaf tag-path and theaddress of t_(i), on the DOM tree. For example, in FIG. 3, a DOM 300 isshown with the root-to-leaf tag-path for “enter” 308 is “<body> 316<table> 314 <tr> 312 <td> 310”. The DOM address of “enter” is“0.0.0.0.1”, which is ordered from top to bottom and from left to rightstarting from 0. FIG. 3 further illustrates other features of thisparticular DOM which is associated with the web page of FIG. 2: searchwords 302, AT&T keywords 304, input 306, local voice 318, products andservices 320 and local solutions 322.

Geometric features such as the vertical and horizontal position wherethis text appears on the web page are also analyzed. There are 119attributes specified in the HTML 4.01 Specification and these attributesare also analyzed. Attributes encode visual features such as the color,font, background color, and size, interactivity features such ashyperlinks and submitting actions, as well as miscellaneous featuresincluding id, style, class and so on. Neighboring nodes also includeanother feature used in the analysis. The leaf nodes between t_(i) andt_(i+1) are denoted as neighboring nodes N_(i). Neighboring nodes oft_(i) include N_(i−1), and N_(i). These nodes are important cues for theweb page semantic structures. They might appear as separators (e.g. anode with the tag <hr>) or interactivity fields (e.g., <input>). Theydirectly affect the semantic role of the neighboring texts. Linguisticfeatures are important cues that humans often use to realize the webpage structure but remain unexplored because of the largedimensionality. In our experiments, we use unigrams and bigrams as thelinguistic features (see table 2 below).

For web page segmentation, the invention involves calculating thedifferences and distances between the above features of two contiguoustext nodes t_(i) and t_(i+1). These features are used as inputs for ourmachine learning algorithms. When using Boostexter, these features arerepresented with text strings, discrete values, continuous values andscored text strings. A more detailed feature list is given in Table 1.When using Llama-SVM, text-string and scored-text-string features arerespectively converted to discrete values and continuous values inadvance.

TABLE 1 Feature Forms Features Continuous DOM-Tree position distanceVertical and horizontal distance Discrete Attribute differences. TextStrings Difference of the root-to-leaf tag-paths Scored Text Leaf nodesbetween t_(i) and t_(i+1.) Strings Difference of neighboring non-textnodes of t_(i) and t_(i+1)

For information block classification, the inventors use the overallproperties of a given information block as the feature set to learn. Asdescribed above, a group of semantic categories (such as 12) are definedfor web page information blocks. Information blocks are denoted startingfrom the text node t_(i) and ending with the text t_(j) node as b_(ij),where j≧i. The features in Table 2 are used to characterize b_(ij).

TABLE 2 Feature Forms Features Continuous The total number of words inb_(ij) The average number of words on each text node The variance of thenumber of words at each text node within The ratio of numeric words inthis information block Geometric Position of b_(ij) The number of textnodes within b_(ij) Text Strings Unigrams and bigrams of the text stringwithin b_(ij) Scored Text All tags used in the nodes within b_(ij)Strings Neighboring nodes of b_(ij) Attributes of the nodes withinb_(ij)

These features are developed based on a variety of heuristicobservations. Machine Learning is expected to make a decisionconsidering all these factors.

In order to learn a general model for discovering web page semanticstructures, a training corpus needs to be built consisting of web pagescovering a broad variety of styles. To serve this purpose, website URLsare collected. The inventors collected 13119 business website URLs andrandomly retrieved 1 web page from each of them. A web page labelingtool was developed through which labelers can conveniently enter theirunderstanding of the semantic structure of a given HTML page. AppendixA, incorporated herein by reference, provides an example of a labelingguide for a labeler. This guide includes example web page structurecategories as well as information on how to use the labeling tool. As aresult of using this labeling tool, each labeled web page is specifiedas a sequence of triplets: {from, to, category}. “from” and “to” areordering numbers of text nodes. “category” is one of the categorieslisted in Table 3.

As part of ongoing work, 2000 web pages in this web page collection hadbeen labeled by three labelers. 32874 information blocks were labeled onthese 2000 pages. Table 3 lists the frequency of information blockcategories in the database for each of these semantic categories.

TABLE 3 Category Frequency Heading-Content 6622 Menu 6215 Heading 5262Other 4823 Normal-Content 3357 Heading-List 1873 Form 1652 Page-Title1177 FAQ-Answer 858 Table-Data 519 Picture-Label 375 Bulletined-List 141

5-fold cross-validation experiments were performed with Boostexter andLlama SVM with polynomial kernels of different degree p.

Table 4 reports the experimental results for web page segmentation withBoostexter and Llama SVM of degree 2 polynomial (Llama-SVM-p2).

TABLE 4 Classification Error (%) 5-Folds Boostexter llama-SVM-p2 1 9.18.6 2 9.1 8.4 3 10.5  9.7 4 9.3 8.2 5 9.0 8.1 Average. 9.4 8.6

Another evaluation measure for boundary detection is based on measuresused in the field of Information Extraction [9] including precision,recall and F1. Precision is the ratio of the correct boundaries amongthe boundaries the classifier produces. Recall refers to the probabilityof correct boundaries that has been detected. F1 combines precision Pand R as follows:

${F\; 1} = \frac{2{PR}}{P + R}$

Table 5 gives the precision, recall and F1 using Boostexter andLlama-SVM-p2. The results are averaged over the five folds.

TABLE 5 Boostexter Llama-SVM-p2 Precision   82% 77% Recall   68% 84% F174.7% 81%

Table 6 reports the experimental results of information blockclassification with Boostexter and Llama SVM of degree 1 polynomial(Llama-SVM-p1).

TABLE 6 Classification Error (%) 5-folds Boostexter Llama-SVM-p1 1 18.916.4 2 18.0 14.7 3 22   11.1 4 20.7 15.3 5 19.8 15.4 Average. 19.8 14.1

These preliminary results encouragingly indicate the effectiveness ofthis proposed learning framework.

FIG. 4 illustrates the method embodiment of the invention. The method ofautomatically analyzing structure for a web page comprises building atraining corpus comprising a broad stylistic coverage of web pages(402), segmenting a web page into information blocks (404), identifyingsemantic categories of the information blocks using the training corpus(406) and applying the identical semantic categories in a web-based tool(408). Other steps in the invention are mentioned throughout thisdisclosure. The steps of segmenting a web page and identifying semanticcategories may be done in a one-pass process or a two-pass process.

Understanding a web page requires recognizing and using web pagestructure. This disclosure proposes an approach to applying machinelearning algorithms including Adaboost and SVMs to web page semanticstructure discovery. This task involves web page segmentation(segmenting a web page into smaller information blocks) and informationblock classification (identifying the semantic categories of thesesmaller information blocks). To achieve this purpose, the inventorsbuilt a training corpus consisting of web pages of a broad stylisticcoverage from the World Wide Web and have these pages manually labeled.Experimental results using 5-fold cross-validation on 2000 labeled webpages show:

Web page information block boundaries can be located with 81% F1 usingpolynomial SVM of degree 2 and 74.7% F1 using Adaboost. Web pageinformation blocks are classified into preferably 12 categories with85.9% accuracy using polynomial SVM of degree 1 and 79% accuracy usingAdaboost. This work can be beneficial to a number of web-drivenapplications such as search engines, web-based question answering,web-based data mining as well as voice enabled web navigation such asvia natural language dialog system. Other web-based tools may alsoutilize the basic features and benefits of the present invention.

Embodiments within the scope of the present invention may also includecomputer-readable media for carrying or having computer-executableinstructions or data structures stored thereon. Such computer-readablemedia can be any available media that can be accessed by a generalpurpose or special purpose computer. By way of example, and notlimitation, such computer-readable media can comprise RAM, ROM, EEPROM,CD-ROM or other optical disk storage, magnetic disk storage or othermagnetic storage devices, or any other medium which can be used to carryor store desired program code means in the form of computer-executableinstructions or data structures. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or combination thereof) to a computer, the computerproperly views the connection as a computer-readable medium. Thus, anysuch connection is properly termed a computer-readable medium.Combinations of the above should also be included within the scope ofthe computer-readable media.

Computer-executable instructions include, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. Computer-executable instructions also includeprogram modules that are executed by computers in stand-alone or networkenvironments. Generally, program modules include routines, programs,objects, components, and data structures, etc. that perform particulartasks or implement particular abstract data types. Computer-executableinstructions, associated data structures, and program modules representexamples of the program code means for executing steps of the methodsdisclosed herein. The particular sequence of such executableinstructions or associated data structures represents examples ofcorresponding acts for implementing the functions described in suchsteps.

Those of skill in the art will appreciate that other embodiments of theinvention may be practiced in network computing environments with manytypes of computer system configurations, including personal computers,hand-held devices, multi-processor systems, microprocessor-based orprogrammable consumer electronics, network PCs, minicomputers, mainframecomputers, and the like. Embodiments may also be practiced indistributed computing environments where tasks are performed by localand remote processing devices that are linked (either by hardwiredlinks, wireless links, or by a combination thereof) through acommunications network. However, any recitation of a “tangible”computer-readable medium expressly excludes non-statutory subject matteras a wired or wireless signed per se. In a distributed computingenvironment, program modules may be located in both local and remotememory storage devices.

Although the above description may contain specific details, they shouldnot be construed as limiting the claims in any way. Other configurationsof the described embodiments of the invention are part of the scope ofthis invention. Accordingly, the appended claims and their legalequivalents should only define the invention, rather than any specificexamples given.

1. A computer-implemented method of automatically analyzing structurefor a web page, the method comprising: building a training corpuscomprising a broad stylistic coverage of web pages, wherein each labeledwebpage of the training corpus is specified as a sequence of {from, to,category} triplets, wherein “from” and “to” are related to an orderingof numbers of text nodes within each respective labeled webpage;segmenting via a processor the web page by grouping all the text nodesof the web page into a sequence of information blocks and identifyingvia the processor semantic categories of the information blocks usingthe training corpus and an adaboost-based text configuration tool,wherein the segmenting and identifying are part of a two-pass process;and applying the identified semantic categories in a natural languagespoken dialog system for voice enabled web navigation.
 2. The method ofclaim 1, further comprising labeling the training corpus either manuallyor automatically using a web page labeling tool.
 3. The method of claim1, further comprises building a feature set representing a distance anddifference between each pair of contiguous text nodes.
 4. The method ofclaim 3, further comprising classifying the feature set into one of aninformation block boundary class or a non-boundary class.
 5. The methodof claim 4, wherein the classifying step is a multi-class classificationstep.
 6. The method of claim 4, wherein a machine learning algorithm isused for the classification step.
 7. The method of claim 4, furthercomprising: identifying related features for each text node in arepresentation model for the web page.
 8. The method of claim 7, whereinthe related features comprise at least one of: a representation modestructure feature; a geometric feature; an HTML attribute; neighboringnodes; and linguistic features.
 9. A system for automatically analyzingstructure for a web page, the system comprising: a processor; a moduleconfigured to control the processor to build a training corpuscomprising a broad stylistic coverage of web pages, wherein each labeledwebpage of the training corpus is specified as a sequence of {from, to,category} triplets, wherein “from” and “to” are related to an orderingof numbers of text nodes within each respective labeled webpage; amodule configured to control the processor to segment the web page bygrouping all the text nodes of the web page into a sequence ofinformation blocks and identify semantic categories of the informationblocks using the training corpus and an adaboost-based textconfiguration tool, wherein the module configured to control theprocessor to segment and identify uses a two-pass process; and a moduleconfigured to control the processor to apply the identified semanticcategories in a natural language spoken dialog system for voice enabledweb navigation.
 10. The system of claim 9, further comprising a moduleconfigured to control the processor to build a feature set representinga distance and difference between each pair of contiguous text nodes.11. The system of claim 10, further comprising a module configured tocontrol the processor to classify the feature set into one of aninformation block boundary class or a non-boundary class.
 12. The systemof claim 9, further comprising a module configured to control theprocessor to identify related features for each text node in arepresentation model for the web page, wherein the related featurescomprise at least one of: a representation mold structure feature; ageometric feature; an attribute; neighboring nodes; and linguisticfeatures.
 13. A tangible computer-readable medium storing instructionsfor controlling a computing device to automatically analyze structurefor a web page, the instructions comprising: building a training corpuscomprising a broad stylistic coverage of web pages, wherein each labeledwebpage of the training corpus is specified as a sequence of {from, to,category} triplets, wherein “from” and “to” are related to an orderingof numbers of text nodes within each respective labeled webpage;segmenting via the processor the web page by grouping all the text nodesof the web page into a sequence of information blocks and identifyingvia the processor semantic categories of the information blocks usingthe training corpus and an adaboost-based text configuration tool,wherein the segmenting and identifying are part of a two-pass process;and applying the identified semantic categories in a natural languagespoken dialog system for voice enabled web navigation.
 14. The tangiblecomputer-readable medium of claim 13, further comprising building afeature set representing a distance and difference between each pair ofcontiguous text nodes.